Biometric camera

ABSTRACT

Exemplary embodiments for a biometric camera system for a mobile device, comprise: a near infrared (NIR) light source on the mobile device that flashes a user of the mobile device with near infrared light during image capture; a biometric camera located on the mobile device offset from the NIR light source, the biometric camera comprising: an extended depth of field (EDOF) imaging lens; a bandpass filter located adjacent to the EDOF imaging lens to reject ambient light during image capture; and an imaging sensor located adjacent the bandpass filter that converts an optical image of an object into an electronic signal for image processing; and a processor configured to receive video images of an iris of a user from the image sensor, and attempt to match the video images of the iris with previously registered images stored in an iris database, wherein if a match is found, the user is authenticated.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Provisional Application Ser. No.61/884,778, filed Sep. 30, 2013; Provisional Application Ser. No.61/885,291, filed Oct. 1, 2013; and Provisional Application Ser. No.61/913,620, filed Dec. 9, 2013, each assigned to the assignee of thepresent application, and incorporated herein by reference.

BACKGROUND

Mobile devices such as laptops—and most recently smartphones—haveincluded biometric devices with the purpose of user identification. Thebiometric devices for example include fingerprint scanners, irisscanners and camera systems capable of face and/or voice recognition.However, fingerprint identification systems suffer from the fact thatusers routinely leave fingerprints on various objects—and thosefingerprints can be “lifted” and reproduced to circumvent fingerprintidentification systems. Also, face and voice recognition systems may notbe sufficiently accurate and can often be circumvented with relativeease. In this situation, iris scanning systems may be of particularinterest because they are relatively more difficult to circumvent, whilebeing very accurate and easy to use.

Various camera systems exist to capture images of the iris. Inparticular, there is interest in providing a biometric camera systemthat can be realized in a mobile device such as a smartphone or tablet.Such biometric camera systems face a number of challenges during theiroperation, including 1) ambient illumination (e.g. be able to operate infull sunlight); 2) motion blur (e.g. user's hand shaking while holdingthe smartphone and capturing iris); 3) depth of field at close distances(e.g. while taking pictures at short (“macro”) distances, the objectbeing further or closer than the focused distance makes the imageblurry); 4) additional cost and additional space for a biometric sensor;and 5) limited field of view (e.g., the camera must capture ahigh-resolution image of the iris for reliable identification. Since theiris is small—e.g. 12 mm in diameter—while the distance between thecamera and the iris is considerable—e.g. arm's length, −25 cm— thecamera must be considerably “zoomed in” on the face and iris area,rather than image the user and his/her surroundings as in a wide-angledshot).

Therefore, a need exists to provide a biometric camera system that iscapable of addressing the above issues.

BRIEF SUMMARY

Exemplary embodiments for a biometric camera system for a mobile device,comprise: a near infrared (NIR) light source on the mobile device thatflashes a user of the mobile device with near infrared light duringimage capture; a biometric camera located on the mobile device offsetfrom the NIR light source, the biometric camera comprising: an extendeddepth of field (EDOF) imaging lens; a bandpass filter located adjacentto the EDOF imaging lens to reject ambient light during image capture;and an imaging sensor located adjacent the bandpass filter that convertsan optical image of an object into an electronic signal for imageprocessing; and a processor configured to receive video images of aniris of a user from the image sensor, and attempt to match the videoimages of the iris with previously registered images stored in an irisdatabase, wherein if a match is found, the user is authenticated.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

These and/or other features and utilities of the present generalinventive concept will become apparent and more readily appreciated fromthe following description of the embodiments, taken in conjunction withthe accompanying drawings of which:

FIG. 1 is a block diagram illustrating an exemplary embodiment of amobile device biometric camera system;

FIG. 2 is a block diagram illustrating components of the biometriccamera for a mobile device according to an exemplary embodiment;

FIG. 3 a diagram illustrating the captured image bounded by a frame inwhich a window of interest containing an iris has been identified;

FIGS. 4A, 4B and 4C are diagrams showing one embodiment in which ashutter is patterned into left and right halves; and

FIGS. 5A and 5B are diagrams showing an example of the shuttersubdivided into more independently addressable areas so the biometriccamera can be used as a plenoptic camera.

DETAILED DESCRIPTION

Reference will now be made in detail to the embodiments of the presentgeneral inventive concept, examples of which are illustrated in theaccompanying drawings, wherein like reference numerals refer to the likeelements throughout. The embodiments are described below in order toexplain the present general inventive concept while referring to thefigures.

Advantages and features of the present invention and methods ofaccomplishing the same may be understood more readily by reference tothe following detailed description of embodiments and the accompanyingdrawings. The present general inventive concept may, however, beembodied in many different forms and should not be construed as beinglimited to the embodiments set forth herein. Rather, these embodimentsare provided so that this disclosure will be thorough and complete andwill fully convey the concept of the general inventive concept to thoseskilled in the art, and the present general inventive concept will onlybe defined by the appended claims. In the drawings, the thickness oflayers and regions are exaggerated for clarity.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the invention (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless otherwise indicated herein or clearly contradicted bycontext. The terms “comprising,” “having,” “including,” and “containing”are to be construed as open-ended terms (i.e., meaning “including, butnot limited to,”) unless otherwise noted.

The term “component” or “module”, as used herein, means, but is notlimited to, a software or hardware component, such as a fieldprogrammable gate array (FPGA) or an application specific integratedcircuit (ASIC), which performs certain tasks. A component or module mayadvantageously be configured to reside in the addressable storage mediumand configured to execute on one or more processors. Thus, a componentor module may include, by way of example, components, such as softwarecomponents, object-oriented software components, class components andtask components, processes, functions, attributes, procedures,subroutines, segments of program code, drivers, firmware, microcode,circuitry, data, databases, data structures, tables, arrays, andvariables. The functionality provided for the components and componentsor modules may be combined into fewer components and components ormodules or further separated into additional components and componentsor modules.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. It is noted that the use of anyand all examples, or exemplary terms provided herein is intended merelyto better illuminate the invention and is not a limitation on the scopeof the invention unless otherwise specified. Further, unless definedotherwise, all terms defined in generally used dictionaries may not beoverly interpreted.

According to the method and system disclosed herein, the exemplaryembodiments provide a mobile device with a biometric camera system forreliable iris identification and user authentication. The camera iscapable of working at high levels of ambient illumination, capturingimages practically without motion blur, provides considerable latitudein distances at which a clear image can be captured and can be realizedin a small form factor. Additionally, the camera can optionally performfunctions including proximity sensing, night-vision camera, 3Dtime-of-flight sensor, eye position and gaze tracking camera, andstructured light for 3D sensing. Combined with a user-facing front-facecamera, the system can automatically identify whether the left and/orright eye is being imaged.

FIG. 1 is a block diagram illustrating an exemplary embodiment of amobile device biometric camera system. The system includes a mobiledevice 10 having conventional components including a memory 12, at leastone processor 14, input output devices (I/O) 16, an front-facing camera18 (e.g., RGB camera), and a display 22.

The memory 12, the processor 14, the I/O 16, the front-facing camera 18and the display 22 may be coupled together via one or more system buses(not shown). The memory 12 may comprise one or more memories comprisingdifferent memory types, including RAM, ROM, cache, virtual memory andflash memory, for example. The processor 14 may include a singleprocessor having one or more cores, or multiple processors having one ormore cores. The I/O 16 is a collection of components that inputinformation and output information. Example components comprising theI/O 16 may include a microphone, speaker, and a wireless networkinterface controller (or similar component) for communication over thenetwork. The processor 14 may execute an operating system (OS) thatmanages hardware resources and performs basic tasks. Examples of the OSmay include Symbian™, BlackBerry OS™, iOS™, Windows™, and Android™. Inone embodiment, the display 22 may be integrated with the mobile device10, while in another embodiment, the display 22 may be external from themobile device 10.

In one embodiment, the mobile device 10 may comprise any type of mobiledevice form factor, including but not limited to, a cell or smart-phone,a tablet, a notebook or laptop computer, a television, and a wearablecomputer, for example. In one embodiment, the mobile device 10 may beimplemented with the display 22 and the front-facing camera 18 locatedon the same side of the mobile device 10, such that the front-facingcamera 18 is pointed at the user has the user holds the device to viewthe display 22. In the embodiment where the mobile device 10 comprises alaptop or notebook, the front-facing camera is typically housed within alid of the laptop.

According to the exemplary embodiment, the mobile device 10 is providedwith a biometric camera system 24 that is used to capture images of thehuman iris for user identification and authentication. In oneembodiment, the biometric camera system 24 may include a near infrared(NIR) light source 20, a biometric camera 28, an iris recognitioncomponent 30 and an iris database 32. In one embodiment, the irisrecognition component 30 and the iris database 32 may be softwarecomponents stored in the memory 12 and executed by the processor 14.

As shown, in one embodiment the biometric camera 28 may be located inone corner of the mobile device 10 (although other locations are alsopossible), while the NIR light source 20 may be located in an oppositecorner to offset the NIR light source 20 from the biometric camera 28within the body of the mobile device 10. In one embodiment, the NIRlight source 20 may be implemented using micro light emitting diodes(LEDs) or a laser diode, and the NIR light source 20 may be slightlyangled upwards to point into the user's eye located at an expecteddistance when held normally by the user.

In operation, the NIR light source 20 flashes a user of the mobiledevice 10 with near infrared light during image capture of the user'siris by the biometric camera 28. Video images output from the biometriccamera 28 are received by the iris recognition component 30, whichattempts to match the image of the iris with previously registeredimages stored in the iris database 32. If a match is found, then theuser may be authenticated.

FIG. 2 is a block diagram illustrating components of the biometriccamera 28 according to an exemplary embodiment. In one embodiment, thebiometric camera 28 may operate to gather reflected near infrared lightand form an image of a human iris using a combination of an imaging lens200, a bandpass filter 202, and an imaging sensor 204. According to theexemplary embodiment, the imaging lens 200 may comprise an extendeddepth of field (EDOF) imaging lens.

The bandpass filter 202 may be located adjacent to the imaging lens 200and functions to reject ambient light. In a preferred embodiment, thebandpass filter 202 may be combined with a solid-state shutter. In oneembodiment, the band pass filter may comprise a near-infrared bandpassfilter. As shown with the dashed lines, the biometric camera 28 may alsoinclude an optional visible light filter 206 located adjacent to thebandpass filter 202 and optional solid-state shutter that may be used tofurther reject visible light.

The imaging sensor 204 may be located adjacent to the bandpass filter202 and converts an optical image of an object (e.g., an iris) into anelectronic signal for subsequent image processing. In one embodiment,the image sensor 204 may include a built-in rolling shutter or afreeze-frame shutter. In one embodiment, the image sensor 204 maycomprise a digital charge-coupled device (CCD) or complementarymetal—oxide—semiconductor (CMOS) active pixel sensors.

In one embodiment, the biometric camera system 28 is designed to have avery small size to fit into a mobile device 10, such as a smartphone,and to also be low in cost. To meet these design goals, in oneembodiment the image sensor 204 may comprise a CMOS sensor with a smallpixel pitch, e.g., 1.1 μm, and a rolling shutter (not freeze-frame).Such a sensor type is commercially available and used in smartphones forfront and back-facing cameras. Thus, the preferred embodiment utilizesan off-the-shelf black-and-white CMOS sensor with rolling shutter andsmall pixels.

The bandpass filter 202 may function to reject all wavelengths exceptnear-infrared, e.g., 800 nm+−5 nm. In a preferred embodiment, thebandpass filter 202 may be combined a solid-state shutter. The processof capturing an image comprises:

-   -   1) Starting the image sensor in a mode where the image sensor is        exposing the entire frame simultaneously;    -   2) Opening the solid-state shutter simultaneously while pulsing        the NIR light source;    -   3) Closing the solid-state shutter; and    -   4) Reading out the captured image from the image sensor.

One goal may be to make the exposure time (when the solid-state shutteris open and the NIR light source is enabled) to be as short aspractically possible. To compensate for the short exposure time, the NIRlight source power may be made as high as practically possible withoutdamaging the human eye. By keeping the solid-state shutter closed at alltimes except during capture, the ambient light received from elsewherein the scene may be significantly reduced. This achieves the goal ofrejecting ambient illumination. Also, due to the very short exposuretime, the goal of practically motion blur-free image capture may beachieved.

In one embodiment, the solid-state shutter may comprise gallium arsenide(GaAs). This type of shutter can change its state from opaque totransparent in a very short time (several nanoseconds) by applyingvoltage externally. Using a GaAs-based shutter as a filter isparticularly desirable in this application since this type of shutterhas a high index of refraction. Typical dichroic (“interference”)optical bandpass filters often have a relatively wide bandpass width dueto the dependence of the central wavelength on the angle of incidence.In camera systems, light rays can pass through the optical filter atvarious angles. Since the bandpass wavelength shifts as a function ofincoming light ray angle, it is typical to make the bandpass wide, suchthat regardless of the incoming ray angle, light energy of the desirablewavelength in that ray can still pass through the optical filter. Theamount of shift of the center wavelength is determined as

$\lambda_{c} = {\lambda_{0}\sqrt{1 - \frac{\sin^{2}\theta}{n^{*2}}}}$

Where λ c is the central wavelength, λ0 is the central wavelength atnormal incidence, and n* is the filter effective index of refraction.Since GaAs has a high n*, the shift effect is reduced, the filter can bedesigned to have a narrower bandwidth—and thus reject even more ambientlight.

Alternatively, instead of using a solid state shutter, a CMOS sensorwith a built-in freeze-frame shutter and NIR bandpass shutter (possiblyalso GaAs) may be used. However, CMOS sensors with a freeze-frameshutter typically have pixels of larger size compared to CMOS sensorsutilizing rolling shutter. Larger pixel size reduces image resolution—orincreases sensor size and cost. Larger pixel size also reduces the depthof field, as explained below. This is why the use of GaAs solid shutteris preferred.

A solid-state shutter, such one based on GaAs, may still leak some lightin its closed state. If the image sensor 204 continues sensing an imagewhile the image is read out, which is typical in image sensors utilizingrolling shutter, the leaking ambient light can register in the capturedimage and adversely affect the image quality, e.g., by saturating someimage areas or adding noise to the image. In one embodiment, theprocessor may be configured to reduce exposure from ambient light byreducing exposure and readout time as follows:

-   -   1) Start exposure of the image sensor 204, in particular in        sensors with a rolling shutter, using a global reset to clear        charge from all pixels simultaneously, thereby reducing time        during which ambient light can be sensed; and    -   2) Reduce the readout time by identifying iris position from a        previously captured image and defining a window of interest        around the iris position, and subsequently capturing and reading        out from a portion of the image sensor defined by the window of        interest. For example, as shown in FIG. 3, a previously captured        image is shown bounded by frame 300, and a window of interest        302 is identified containing the iris, which is smaller than the        entire frame.

In one embodiment, the NIR light source may be synchronously pulsed withthe readout of the rolling shutter. In this embodiment, the rollingshutter may be addressed in a row-column manner where only the pixels inthe window of interest 302 are read out. For example, the NIR lightsource may turned on at or slightly before the time where the upper leftof the window of interest 302 is read out and then turned off after thelower right pixel of the window of interest 302 has been read. In aparticular embodiment, the rejection of ambient light can be madegreater by pulsing at a higher light intensity. The total andinstantaneous amounts of NIR light must be below certain thresholds toavoid eye damage. Because the NIR light is only turned on while exposingand reading the window 302, the intensity of the light may be increasedduring this period than would be allowed for eye safety if it were usedwhile exposing and reading the entire frame 300. The increased intensityof light enables a shorter exposure and readout time, leading to lessambient light readout.

In one embodiment, a rolling shutter is used to read entire lines ratherthan a small window 302. In this case, the shutter resets the pixels andreads out at the top of the window 302 and finishes reading at thebottom of the window. In one embodiment, the NIR light may be turned onand off to synchronize with the beginning and end of this readout. Asthis slice of the frame 300 is smaller than the full frame, theintensity of the pulsed NIR may be higher than if exposing the fullframe. In a particular embodiment, the NIR light may be further pulsedto synchronize with each line readout. Here, although the full lines arebeing read out, the NIR is only turning on while exposing the window ofinterest 302. The instantaneous intensity may be made even higher inthis embodiment than in the previous one, even though the total NIR fluxmay be reduced.

Depth of Field

The biometric camera 28 should typically capture an iris image atdistances of around 25 cm. At such short distances depth of field istypically limited and it is difficult to obtain a sharp image. Toincrease the depth of field, the exemplary embodiments may include:

1) Using EDOF (extended depth of field) techniques, in particularwavefront coding type of EDOF, preferably combined with a fixed-focuslens;

2) Reducing lens aperture, while boosting light source power tocompensate;

3) Reducing pixel size, while boosting light source power to compensate;and

4) Using an auto-focus actuator. Although the main purpose of theauto-focus actuator is to focus speedily on objects at “macro” range,e.g. around 10-50 cm, for iris scanning, the actuator may also focus atlonger distances up to infinity for applications described below.

It should be noted, EDOF systems were historically utilized for colorcapture and experienced some issues capturing and correcting imagesacross all visible wavelengths, resulting in degradation of imagequality. Since the present embodiments are capturing a monochromatic NIRimage, the issues related to operation across a wide range ofwavelengths are eliminated and system performance can improve.Additionally, unlike color sensors typically utilizing Bayer patternwhere only one of three colors is sampled at each pixel location, themonochrome sensor samples the image at every pixel location, thusproviding EDOF with full information for improved system performance.

Although EDOF systems increase the depth of focus to improve the rangewhere iris pictures may be taken, they also tend to decrease the MTF ofthe picture. In one embodiment, the image quality may be improved byusing known super-resolution techniques to capture several images of theiris and process them to achieve a sharper picture. To achieve this,several pictures may be taken in succession with the sub-window rollingshutter to quickly capture the window of interest 302 multiple times. Ifthere is movement between the frames, image registration can be donebefore combining. The combining step may use a super-resolutiontechnique, or could instead do a simple signal averaging to reducenoise. In a particular embodiment, this may be combined by synchronizingthe NIR pulse with reading out of the window of interest 302 to achievefaster readouts of the window and ameliorating effects of movementwithin the frame.

Additional Functions

In a further aspect of the exemplary embodiment, the architecture of thebiometric camera system 24 enables the biometric camera system 24 toperform additional functions in addition to iris scanning, includingproximity sensing, night-vision, 3D time-of-flight sensing, eye positionand gaze tracking, and providing a structured light for 3D sensing, asfurther described below.

Proximity Sensor

Proximity sensors utilize an imaging sensor and a NIR light source. Theimage sensor typically comprises a single pixel (not array of multiplepixels) and the proximity operation may comprise:

-   -   1) Capturing a first signal with the image sensor and measuring        a first intensity of ambient light;    -   2) Enabling the NIR light source;    -   3) Capturing a second signal with the image sensor and measuring        a second intensity of ambient and any reflected NIR light from        the object/subject in proximity;    -   4) Disabling the NIR light source; and    -   5) Determining a difference between the first and second        intensity of ambient light measurements, where a signal        difference surpassing a threshold indicates an object present in        proximity. In the exemplary embodiment, the sensor with shutter        and NIR light captures pictures with and without an NIR light        source flash, (optionally) sums pixel outputs in each image and        subtracts the two values.

Note that proximity sensors are typically designed to have a highdynamic range, such that they work outdoors in high ambient light. Whena GaAs shutter is used, the shutter can open for a very short time(synchronized with the NIR light source pulsing) and thus greatly reducethe exposure from the ambient light. Therefore, the high dynamicfunctionality may not be necessary, thus potentially reducing imagercost and increasing image quality.

Typically, every smartphone is equipped with a proximity sensor to turnoff the touch screen while the user is holding the phone against his/herear to prevent accidental touch input. Thus, importantly, since thebiometric camera system 24 may perform the function of the proximitysensor, the proximity sensor may be removed from the phone, and replacedwith the biometric camera system 24 without sacrificing thefunctionality and without unnecessarily growing smartphone size andcost. Exemplary embodiments also may reuse an already-existing NIR lightsource for both proximity sensing and biometric scanning.

Night Camera

Since the biometric camera 28 utilizes NIR and an NIR light source 20,the biometric camera 28 can capture images in darkness. The system 24can also detect a user in total darkness, scan an iris, capture usergestures and so on. Utilizing an auto-focus actuator instead of afixed-focus (preferably EDOF) lens may be beneficial to be able toclearly capture objects distances further than 25 cm.

3D Time-of-Flight Sensor

An off-the-shelf CMOS camera equipped with a fast GaAs shutter and NIRlight source is capable of capturing distance images. For example, atime-of-flight camera is a class of scannerless LIDAR, in which theentire scene is captured with each laser or light pulse. It should benoted that the distance measured to an object (e.g., a face) by thetime-of-flight camera can be used for fast focusing using the auto-focusactuator.

Eye Position Detection and Gaze Tracking Camera

The availability of NIR light source 20 and NIR biometric camera 20allows capturing iris and eyes continuously to track their location andgaze direction using existing techniques.

Motion Detection

The NIR biometric camera 28 can be used to detect motion, includingmotion in complete darkness, by utilizing imaging techniques known inthe field (e.g., capturing frames and comparing their aggregatedsharpness).

Structured Light for 3D Sensing

A patterned slide can be inserted in front of the NIR light source 20 toimplement 3D sensing. The pattern will be reflected off the object beingimaged and then captured by the biometric camera 28. By comparing therelative distortion of the pattern between imager and projected version,the absolute distance of points on the object can be determined.

Typically structured light capture can only be used for imaging staticscenes at high resolution because moving objects change position toorapidly to update the depth map without blur. Furthermore, in brightambient light conditions, structured lighting depth capture systemssuffer from wash out.

In this system, the fast shutter and a strong NIR light source 20 canmitigate against both of these issues. The problem of ambient wash outmay be reduced if another NIR light source is added with a patternedslide that has a complementary pattern to the pattern on the firstslide. For example, if the new slide has a pattern inverse to the firstpattern, then an object can be rapidly illuminated with the first NIRlight source and then the second, synchronized with the fast shutter,and then create a higher resolution pattern by using standardcomputational imaging operations on the two captured patterns. Thiswould enable, e.g., 3D video capture, or the ability to track smallchanges in facial expression of a person holding the camera infront-facing mode.

3D Face Recognition for Biometrics

In the previous embodiment described above, the system can use the fastshutter and a structured light pattern over the IR illuminator tocapture high precision, low blur, 3D depth maps of a person's face,paired with the RGB images of the face, captured from the standardfront-facing camera. In this embodiment, the field of view of the NIRcamera is designed to be wide enough to capture the entirety of theuser's face at the target distance range for the application (typically10 cm-25 cm). These 3D depth maps and RGB image pairs can be used tocreate a 3D model of a person's face, which can then be used to create abiometric signature, unique to that user. A benefit of this embodimentis that the same system setup can be used to simultaneously capture theperson's iris image and 3D facial image, allowing for multi-factorauthentication. Capture of the iris and 3D face may be done with thesame IR image, or there may be possibly two IR illuminators, one withthe structured light pattern on top of it and the other with no patternon top. This can potentially simplify the processing of the image whichwould be needed to remove the structured light pattern from the irisimage. The same camera could be used in rapid alternating sequence tocapture iris and 3D face images.

Capture with Addressable Shutter

In some previous embodiments, the solid-state shutter of the biometriccamera 28 is addressable with a single set of electrodes and covers theentire field of view of the shutter. In one embodiment, electrodes maybe patterned on the shutter so that there are two or more separatelyaddressable areas of the shutter, that is, one portion of the shuttercould be left open to the wavelengths of light measured while the otherportion or portions could remain closed. FIGS. 4A, 4B and 4C arediagrams showing one embodiment in which a shutter 400 is patterned intoleft and right halves. FIG. 4B is a diagram showing the left half 402,and FIG. 4C is a diagram showing the right half 404. Here, alternatepictures can be taken with each half successively blocked while theother side remains open. For example, a 3D stereo image may be capturedthrough single lens technology with time-multiplex. A left stereo pairimage may be captured with the left half 402 of the shutter 400transparent and the right half 404 opaque. A right stereo pair image maybe captured with the right half 404 of the shutter 400 transparent andthe left half 402 opaque. The left and right captured images can then beprocessed using known techniques to provide stereo pair images using asingle lens camera. In this application, strong ambient lighting mightbe used to capture the images. In a particular embodiment, the NIR lightsource may also be used to supplement or replace ambient light. The NIRpulses may be synchronized to the timing of the shutter halves whileincreasing exposure lighting without exceeding eye safety limits.

In another embodiment, the shutter of the biometric camera 28 may befurther subdivided into more independently addressable areas to create apatterned shutter with time-multiplex for the capture of light-fieldphotos. FIGS. 5A and 5B are diagrams showing an example of the shutter500 subdivided into more independently addressable areas 502 so thebiometric camera can be used as a plenoptic camera. In this example,successive frames may be taken, each with on of the areas 502 of theshutter 500 opened during its frame capture. The series of capturedframes can then be processed as a light field image set. Although FIG.5B shows an example of only a single addressable area 502 open at onetime for a frame capture, in another embodiment, multiple areas 502 ofthe shutter could be open at the same time in various patterns.

Combined Use with RGB Camera

Since the biometric camera 28 may have a relatively narrow field of view(˜30 degrees horizontal), it may not always capture the image of theentire user face, but only an image of one eye. To decide which eye thisis—left or right—the output of the front-facing camera 18 may becombined to determine which of the eyes is being imaged by the biometriccamera 28. The image from the front-facing camera 18 may provideinformation regarding the position of the face with respect to the fieldof view imaged by the biometric camera 28.

Other combined uses include:

1) The time-of-flight operation can be used to assist the front-facingcamera 18 for focusing;

2) The front-facing camera 18 can also automatically capture user's faceduring the iris scan, e.g. for reference purposes;

3) The front-facing camera 18 can capture the user's face toconcurrently with the iris scan to additionally verify that the user isa living human (e.g. by tracking pulse using known methods based onspatio-temporal filtering and amplification of color);

4) The front-facing camera 18 may confirm the presence and determine theposition of the user's head and eyes during iris scan to assistrecognition in case of problems with head positioning, such that thesystem can guide the user to position his/her head properly (and alsoe.g. by showing the user a target to look at on the display or e.g.flash LED on the phone); and

5) The front-facing camera can detect the presence of glasses, hair andidentify other problems preventing successful iris scan. The system(e.g., smartphone) can then direct the user to address these problems,e.g. remove glasses, adjust hair and track and verify that the problemsare addressed using the front-facing camera.

A method and system for a biometric camera system has been disclosed.The present invention has been described in accordance with theembodiments shown, and there could be variations to the embodiments, andany variations would be within the spirit and scope of the presentinvention. For example, the exemplary embodiment can be implementedusing hardware, software, a computer readable medium containing programinstructions, or a combination thereof. Software written according tothe present invention is to be either stored in some form ofcomputer-readable medium such as a memory, a hard disk, or a CD/DVD-ROMand is to be executed by a processor. Accordingly, many modificationsmay be made by one of ordinary skill in the art without departing fromthe spirit and scope of the appended claims.

We claim:
 1. A biometric camera system for a mobile device, comprising:a near infrared (NIR) light source on the mobile device that flashes auser of the mobile device with near infrared light during image capture;a biometric camera located on the mobile device offset from the NIRlight source, the biometric camera comprising: an extended depth offield (EDOF) imaging lens; a bandpass filter located adjacent to theEDOF imaging lens to reject ambient light during image capture; and animaging sensor located adjacent the bandpass filter that converts anoptical image of an object into an electronic signal for imageprocessing; and a processor configured to receive video images of aniris of a user from the image sensor, and attempt to match the videoimages of the iris with previously registered images stored in an irisdatabase, wherein if a match is found, the user is authenticated.
 2. Thesystem of claim 1, wherein the bandpass filter comprises a near infraredbandpass filter.
 3. The system of claim 1, wherein the bandpass filteris combined with a solid-state shutter.
 4. The method of claim 3,wherein the solid-state shutter comprises gallium arsenide (GaAs). 5.The method of claim 3, wherein the solid-state shutter has at least twoseparately addressable shutter areas, such that successive frames may becaptured, each with at least one of the separately addressable shutterareas opened during the frame capture.
 6. The system of claim 1, furthercomprising a visible light filter located adjacent to the bandpassfilter that rejects visible light.
 7. The system of claim 1, wherein theimage sensor further includes at least one of a rolling shutter and afreeze-frame shutter.
 8. The system of claim 1, wherein the processor isfurther configured to reduce exposure from ambient light by: startingexposure of the image sensor; and reducing readout time by identifyingan iris position on a user from a previously captured image and defininga window of interest around the iris position, and subsequentlycapturing and reading out from a portion of the image sensor defined bythe window of interest.
 9. The system of claim 1, wherein the imagesensor comprises a rolling shutter and wherein the NIR light source issynchronously pulsed with the readout of the rolling shutter.
 10. Thesystem of claim 1, wherein the processor is further configured toimplement a proximity sensor operation by: capturing a first signal withthe image sensor and measuring a first intensity of ambient light;enabling the NIR light source; capturing a second signal with the imagesensor and measuring a second intensity of ambient light and anyreflected NIR light; disabling the NIR light source; and determining adifference between the first and second intensity of ambient lightmeasurement, wherein a signal difference surpassing a thresholdindicates an object present in proximity.
 11. The system of claim 1,wherein the NIR light is covered with a structured light pattern toimplement 3D sensing.
 12. The system of claim 1, wherein the NIR lightsource and the biometric camera are used for at least one of: a nightcamera, a 3D time-of-flight sensor, eye position detection and gazetracking; and motion detection.
 13. A biometric camera system for amobile device, comprising: a front-facing camera; a near infrared (NIR)light source on the mobile device that flashes a user of the mobiledevice with near infrared light during image capture, the NIR lightsource being covered with a structured light pattern; a biometric cameralocated on the mobile device offset from the NIR light source, thebiometric camera comprising: an extended depth of field (EDOF) imaginglens; a bandpass filter located adjacent to the EDOF imaging lens toreject ambient light during image capture; and an imaging sensor locatedadjacent the bandpass filter that converts an optical image of an objectinto an electronic signal for image processing; and a processorconfigured to: use the front-facing camera to capture RGB images of auser's face; use the NIR light source with the structure light patternand the biometric camera to capture one or more 3D depth maps of theuser's face; use the RGB images and the one or more 3D depth maps tocreate a 3D model of the user's face; and use the 3D model to create abiometric signature for the user.
 14. The system of claim 13, whereinthe biometric camera system is used to simultaneously capture an imageof the user's iris to perform multi-factor authentication based on irisdetection and 3D model detection.
 15. The system of claim 13, whereinthe bandpass filter comprises a near infrared bandpass filter.
 16. Thesystem of claim 13, wherein the bandpass filter is combined with asolid-state shutter.
 17. The method of claim 16, wherein the solid-stateshutter comprises gallium arsenide (GaAs).
 18. The method of claim 16,wherein the solid-state shutter has at least two separately addressableshutter areas, such that successive frames may be captured, each with atleast one of the separately addressable shutter areas opened during theframe capture.
 19. The system of claim 13, further comprising a visiblelight filter located adjacent to the bandpass filter that rejectsvisible light.
 20. The system of claim 13, wherein the image sensorfurther includes at least one of a rolling shutter and a freeze-frameshutter.